This disclosure generally relates to the display of synchronized side-by-side viewing of photographic and computer-generated images of complex objects such as aircraft during design, assembly or maintenance.
When building or maintaining a complex object like an aircraft, it is useful to have access to photographic imagery (e.g., panoramic images) showing the assembled or partially assembled object. But this “as-built” representation takes time to acquire and may not be available for all instances of the target object (especially instances that are not built yet).
As used herein, the term “3-D model” means a 3-D computer-aided design (CAD) or other similar model (sometimes referred to as a solid model). A three-dimensional (3-D) model visualization system may be used to display images representing portions and/or individual components of one or more 3-D models of a complex object within a graphical user interface. Visualization systems may be used to perform various operations with respect to the image of the object. For example, a user may use a 3-D model visualization system to navigate to an image of a particular part or assembly of parts within an object for the purpose of identifying information for use in performing an inspection or a repair. Visualization of 3-D models can provide an “as-designed” representation of the target object or environment and can be created before the target object exists, but changes or updates that occur during assembly might not be available in this representation of the target object.
In the aviation industry, users who need visual information about an aircraft typically walk to the aircraft to see how some portion of the aircraft was assembled or try to determine information from two-dimensional (2-D) drawings. Three-dimensional model visualization systems are also available, but do not include photorealistic representations of the scene.
Existing photographic image-based viewer applications, such as “virtual tour” applications that use spherical panoramic images and spherical image viewers, improved this situation by allowing users to view real-world panoramic images of a previous version of the aircraft from multiple angles on a personal computer (PC), laptop, or tablet—but they could not display the current version of the aircraft. Here we will be using the words “panoramic image” and “panoramic image viewer application” to represent the class of photographic image-based viewer applications that use one or more curvilinear or stitched rectilinear photographic images to provide a 360-degree representation of the target environment, a portion of which will be shown to the user in the form of a transformed rectilinear image in the image viewer application. In addition, an existing 3-D visualization application also allows users to see models of the nominal designs for specific line number aircraft in context, but the images generated by the 3-D visualization are not photorealistic and may miss some of the subtle features of the actual aircraft.
While existing virtual tour types of applications provide as-built visual data, that data is captured for a specific line number aircraft, which may not be exactly how subsequent (or prior) aircraft look. Also, parts could not be removed from the digital photographs to show what is behind them. In contrast, the existing 3-D visualization application has the ability to show 3-D data for each specific line number aircraft, but cannot show the as-built environment, which also has a visual difference (since the images rendered in the 3-D visualization are non-photorealistic).
It would be desirable to provide an application that addresses the shortcomings of each of the above-described approaches by providing a method for combined use of data from physical (i.e., photographic) images and 3-D models.